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DETAILED ACTION 

Response to Arguments 

Duplicate Claims, Warning 

Applicant's arguments, with respect to the warning of duplicative claims have been fully 
considered and are persuasive. There was a typo regarding the duplicative claims as specified 
in the Office Action 12/18/2008. Claims 8 and 9 should be mentioned in the warning, instead of 
claims 7 and 9. The warning of duplicative claims 7 and 9 has been withdrawn. However, a new 
warning of duplicative claims 8 and 9 will be addressed in this Office Action. 

Claim Rejections - 35 USC $ 101 

The rejection of claims 22, 32 and 51 under 35 U.S.C. § 101 has been withdrawn in view 
of the amendments. 

Claim Rejections - 35 USC § 1 12 

Applicant's arguments with respect to the rejection of claims 1, 33, 36 and 51 under 35 
U.S.C. § 112, 1 st paragraph, and the rejection of claim 33 under 35 U.S.C. § 112, 2 nd paragraph, 
have been fully considered and are persuasive. The rejections of claims 1, 33, 36 and 51 under 
35 U.S.C. § 1 12, 1 st paragraph, and claim 33 under 35 U.S.C. § 1 12, 2 nd paragraph, has been 
withdrawn. 

Claim Rejections - 35 USC § 102 and 103 

• Applicant's arguments with respect to the rejection of claim 1 under 35 U.S.C. § 
102(b) have been fully considered but they are not persuasive. 

As argued by applicant (Remarks, Page 18): 
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The rejection of claim I under 35 U.S.C. § 102(b) over Graham should be w ithdrawn 
because Graham does not expressly nor inherently disclose each and every element of the 

For example. Graham does not expressly nor inherently disclose "comparing the 
candidate object interval to a weighted measure of cluster extent for the current object cluster, 
the measure of cluster extent corresponding to a current distance spanned by all the objects in 
the current object cluster measured in the selected dimension of the context-related 
metadata, " as recited in claim 1. 

The examiner respectfully disagrees. 

As taught by Graham, when taking a picture with a digital camera, the equipment 
records the date and time when the photo was taken, as well as technical details, such as 
aperture settings, distance from the focal plane, and whether a flash was used. This data is 
encoded into the image files (Graham, Page 2, Col. 1-Lines 32-37). In order to cluster images 
based on time metadata, the Graham's technique has three steps: (1) Create initial cluster, (2) 
Split initial cluster and (3) Create parent cluster. Creating initial cluster is processed by sorting 
the list of photographs in the collection with respect to time. Root node and initial clusters under 
root node are created by iterating the sorted list of photographs for adding photographs into 
initial clusters using a specified constant time difference between two consecutive photographs, 
e.g., every time two consecutive photographs differ by more than a specified constant time 
difference, the current image is added to a new initial cluster (Page 4, Col. 2-Lines 3-18). 

In view of Graham's teaching, for example, a list of photographs is sorted by recorded 
time as below: 

T1 (7:30Am)-T2 (7:32Am)-T3 (9:30 Am)-T4 (9:45Am)-T5 (10:07Am)... 
wherein T1, T2... represent photographs, and each photograph has a corresponding 
recorded time. 
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If greater than 60 minutes is specified as a constant time difference between two 
consecutive photographs as taught by Graham, a first initial clusters C1 will be created for {T1 , 
T2}, because the difference between T1-T2 is 2 minutes and less than 60 minutes, wherein the 
difference between T2 and T3 is 1 18 minutes. T3 will be processed against T4 and in the same 
manner to the end of the sorted list. 

The clustering in Graham's technique is based on time from photo's metadata. Time is 
considered as being equivalent to the claimed selected dimension of context-related metadata. The 
Graham's time interval between two consecutive photographs is considered as being equivalent 
to the claimed candidate object interval. The Graham's specified constant time difference, e.g., 
greater than 60 minutes, is considered as being equivalent to the claimed a weighted measure of 
cluster extent. The maximum time difference of every two consecutive objects in a cluster, e.g., 
equal or less than 60 minutes of T1-T2 in C1, is considered as being equivalent to the claimed 
the measure of cluster extent. The maximum time difference of T1 and T2 corresponds to a current 
distance, e.g., 60 minutes, spanned by all the objects in the current object cluster, e.g., equal or less than 
60 minutes distance is spanned from every consecutive photographs in C1. The maximum 60 
minutes is time measurement or the selected dimension of context-related metadata. 

In short, the Graham's technique reads on the claimed limitation comparing the candidate 
object interval to a weighted measure of cluster extent for the current object cluster, e.g., time interval 
between two consecutive photographs such as T1 and T2 is compared with specified constant 
time difference such as greater than 60 minutes for the current cluster C1 , the measure of cluster 
extent, e.g., the maximum time difference of every two consecutive objects such as T1-T2 in C1, 
corresponding to a current distance, e.g., the maximum time difference of every two consecutive 
objects such as T1-T2 corresponds to equal of less than 60 minutes distance, spanned by all the 
objects in the current object cluster, e.g., equal or less than 60 minutes distance is spanned from 
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every consecutive photographs in C1 , measured in the selected dimension of the context-related metadata, 
e.g., the maximum 60 minutes is time measurement. 

• Applicant's arguments with respect to the rejection of claims 2-19, 21 and 22 
under 35 U.S.C. § 102(b) have been fully considered but they are not persuasive (Remarks, 
Page 19). Claims 2-19, 21 and 22 are unpatentable over Graham for at least the reasons as 
discussed above regarding claim 1. 

• Applicant's arguments with respect to the rejection of claim 23 under 35 U.S.C. § 
102(b) have been fully considered but they are not persuasive. 

As argued by applicant (Remarks, Pages 19-20): 

The rejection of claim 23 under 35 U.S.C. § 102(b) over Graham should be withdrawn because 
Graham does not expressly nor inherently disclose each, and every element of the claim. 

For example, Graham does not expressly nor inherently disclose "extracting context-related meta data 
corresponding to object generation locations associated with the objects unci parsable into multiple levels 
of a name hierarchy. " The "corresponding to object generation locations" element of claim 23 originally 
was recited in dependent claim 26. In support of the rejection of claim 26, the Examiner has taken the 
position that Graham discloses the "corresponding to object generation locations" element of claim 23 on 
page 4, col. 2, lines 33-42 (see page 12: Sixth f of the Office action) Contrary to the Examiner's assertion, 
however, page 4. col. 2, lines 33-42, only discloses the process of refining the initial clusters based on the 
rale Lit which photographs are taken. The description on page 4. col. 2. lines 33-42, relating to different 
locations merely illustrates examples of cases in which "the rale at which photographs are taken is likely to 
differ significantly between types of events. " 

For at least this reason, the rejection of independent claim 23 under 35 U.S.C. § 102(b) over Graham 
should he withdrawn. 

The examiner respectfully disagrees. 

As taught by Graham, as global position system (GPS) are integrated into digital 
cameras, the location where a photograph was taken could also be extracted from the image 
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files and introduced as a clustering criterion (Graham, Page 3, Col. 2-Line 39->Page 4, Col. 1- 
Line 2). The cluster engine uses the extracted metadata to compute clusters. It is equally easy 
to cluster the data based on location when GPS data is available. The engine's output is a 
cluster tree (Graham, Page 4, Col. 1 -Lines 8-13). The generated clusters of photographs are 
combined into more general clusters using fixed year-month-day hierarchy for the higher levels. 
That is, for each cluster C discovered in step 1, date of its first photograph is determined and 
then C is made as a child of the appropriate day in the year-month-day hierarchy (Graham, 
Page 5, Col. 1-Lines 16-29). 

The Graham's teaching as discussed reads on the claimed limitation extracting context- 
related meta data corresponding to object generation locations associated with the objects, e.g., GPS location 
data associated with photographs is extracted, and parsable into multiple levels of a name hierarchy, 

e.g., the GPS location data is analyzed as a clustering criterion into year-month-day hierarchy. 

• Applicant's arguments with respect to the rejection of claims 27-32 under 35 
U.S.C. § 102(b) have been fully considered but they are not persuasive (Remarks, Page 20). 
Claims 27-32 are unpatentable over Graham for at least the reasons as discussed above 
regarding claim 23. 

• Applicant's arguments with respect to the rejection of claim 33 under 35 U.S.C. § 
102(b) have been fully considered but they are not persuasive. 

As argued by applicant (Remarks, Page 21): 

The rejection of chum 33 trader 35 U.S.C. $ 1 02(b) over Graham should be withdrawn because 
Graham does not expressly nor inherently disclose each and every element of the claim. 

For example, Graham does not expressly nor inherently disclose "selecting for each object cluster at 
least two constituent objects representative of beginning and ending instances in the corresponding object 
sequence. " 
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The Examiner has taken the position that Graham discloses this element of claim 33 on piige 5. col. 2, 
lines 42-47 (see page 14, top\ of the Office action). Contrary to the Examiner's position, however, the cited 
paragraph of Graham only discloses how the screen space is allocated in proportion to the number of 
photos in each cluster. 

/•'oral least this reason, the rejection of claim 33 under 35 U.S.C. $ 102(h) over Graham should he 
withdrawn. 

The examiner respectfully disagrees. 

As taught by Graham in the section Creating Summaries on page 5, an alternative to 
scrolling is summarization as in FIG. 1 a and 1 b. Instead of displaying all images, a set of 
representative images is shown (Graham, Page 5, Col. 2-Lines 14-16). The input to Graham's 
summarization procedure is a set of sequential cluster C at level k in the hierarchy and a target 
T, the desire number of representative photographs (Graham, Page 5, Col. 2-Lines 23-25). The 
summarization procedure assigns one space to each cluster in C. If the total of one space to 
each cluster is M, the remaining T - M spaces is assigned to clusters in proportion to their sizes. 
As an example of creating the summarization of 3 clusters, C1 (including its children) has 20 
photos, C2 has 10 and C3 has 5. After one slot is assigned for each cluster, the remaining 7 
slots are assigned proportionally to C1-C3, e.g., 4 more to C1, 2 to C2 and 1 to C1. Thus, C1 
gets 5 spaces, C2 get 3 and C3 get 2 (Graham, Page 5, Col. 2-Lines 35-47). To select 
summarization photographs for the assigned spaces, representative images are chosen by 
looking for the smallest time difference and largest time differences between consecutive 
photographs. For example, if 4 summary photos are needed, 1 of closest photos and 3 of 
largest difference are selected (Graham, Page 6, Col. 1-Lines 37-49). 

In view of Graham's teaching of assigning spaces for C1 , C2 and C3 and selecting 
summarization photographs for assigned space, C1 has 1 photograph representing the smallest 
time difference and 4 photos representing the largest time differences. C2 has 1 photograph 
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representing the smallest time difference and 2 photos representing the largest time differences. 
C3 has 1 photograph representing the smallest time difference and 1 photo representing the 
largest time difference. 

The Graham's teaching as discussed indicates the step oft selecting for each object cluster at 
least two constituent objects representative of beginning and ending instances in the corresponding object 
sequence, e.g., for C3, at least 2 constituent photos (1 photograph representing the smallest time 
difference and 1 photo representing the largest time difference) representing beginning and 
ending instances in the corresponding photo sequence of C3, which is 5 photos. 

• Applicant's arguments with respect to the rejection of claims 34-37, 39-49 and 51 
under 35 U.S.C. § 102(b) have been fully considered but they are not persuasive (Remarks, 
Page 21). Claims 34-37, 39-49 and 51 are unpatentable over Graham for at least the reasons 
as discussed above regarding claim 33. 

• Applicant's arguments with respect to the rejection of claims 20, 38 and 50 under 
35 U.S.C. § 103(a) have been fully considered but they are not persuasive (Remarks, Page 21). 
Claims 20, 38 and 50 are unpatentable over Graham for at least the reasons as discussed 
above regarding claims 1 and 33. 

• In view of the foregoing reasons, the rejections under 35 U.S.C. § 1 02(b) and 
103(a) are continued as following. 
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Duplicate Claims, Warning 

Applicant is advised that should claim 8 be found allowable, claim 9 will be objected to 
under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application 
are duplicates or else are so close in content that they both cover the same thing, despite a 
slight difference in wording, it is proper after allowing one claim to object to the other as being a 
substantial duplicate of the allowed claim. See MPEP § 706.03(k). 

Claim Rejections - 35 USC §102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1-19, 21-23, 27-37, 39-49 and 51 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Graham et al. [Time as Essence for Photo Browsing Through 
Personal Digital Libraries]. 

Regarding claims 1 and 22, Graham teaches a method and system for organizing a 
collection of objects arranged in a sequence ordered in accordance with a selected dimension 
of context-related metadata respectively associated with the objects (A collection of images 
arranged in is sorted with respect to time as a selected dimension of context-related metadata 
respectively associated with the objects (Graham, Page 4 1 , Col. 2 2 Lines 4-5)), comprising: 



1 The first page of the reference is considered as page 1. 

2 The left column of the page is col. 1 and the right column of the page is col. 2. 
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a computer-readable medium storing computer-readable instructions; and a data processing; unit 
coupled to the memory, operable to execute the instructions (These features are inherited features of a 
computer system as disclosed by Graham on Page 5, Col. 2-Lines 2-13), and based at least in 
part on the execution of the instructions operable to perform operations comprising 

classifying the objects in the sequence to generate a series of object clusters, wherein the classifying 
comprises sequentially processing each of the objects as a respective candidate for segmentation into a respective 
current one of the object clusters in the series and, for each of the candidate objects (A series of object 
cluster as in Fig. 4 (Graham, Page 5) is generated according to classifying technique as 
disclosed on Page 4, Col. 2 Lines 3-18), 

determining a candidate object interval separating the candidate object from an adjacent object in the 
sequence already segmented into the current object cluster, the candidate object interval being measured in the 

selected dimension of the context-related metadata (As further disclosed by Graham, for each image in 
the sorted list, time interval of two consecutive images is determined, the time interval separates 
two consecutive images, wherein one is a candidate image and one is an adjacent image in a 
current cluster (Graham, Page 4, Col. 2 Lines 3-18). The time interval as disclosed by Graham 

is Considered as being equivalent to candidate object interval separating the candidate object from an 
adjacent object in the sequence already segmented into the current object cluster. The time interval is 
measured in time as selected dimension of context-related metadata respectively associated with the objects), 
comparing the candidate object interval to a weighted measure of cluster extent for the current object 
cluster, the measure of cluster extent corresponding to a current distance spanned by all the objects in the 
current object cluster measured in the selected dimension of the context-related metadata (Every time two 
consecutive photographs differ by more than a specified constant time difference, e.g., 1 hour 
difference, a new cluster is created and the current image is added to the most recently created 
cluster (Graham, Page 4, Col. 2 Lines 3-28). In view of Graham's teaching, for example, a list of 
photographs is sorted by recorded time as: T1 (7:30Am)-T2 (7:32Am)-T3 (9:30 Am)-T4 
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(9:45Am)-T5 (10:07Am)... wherein T1, T2... represent photographs, and each photograph has 
a corresponding recorded time. If greater than 60 minutes is specified as a constant time 
difference between two consecutive photographs as taught by Graham, a first initial clusters C1 
will be created for {T1 , T2}, because the difference between T1-T2 is 2 minutes and less than 
60 minutes, wherein the difference between 12 and T3 is 1 18 minutes. T3 will be processed 
against T4 and in the same manner to the end of the sorted list. In short, the Graham's 

technique reads On the Claimed limitation comparing the candidate object interval to a weighted measure of 

cluster extent for the current object cluster, e.g., time interval between two consecutive photographs 
such as T1 and T2 is compared with specified constant time difference such as greater than 60 
minutes for the current cluster C1, the measure of cluster extent, e.q., the maximum time difference of 
every two consecutive objects such as T1-T2 in C1 , corresponding to a current distance, e.g., the 
maximum time difference of every two consecutive objects such as T1-T2 corresponds to equal 

Of less than 60 minutes distance, spanned by all the objects in the current object cluster, e.g., equal Or 

less than 60 minutes distance is spanned from every consecutive photographs in C1, measured in 

the selected dimension of the context-related metadata, e.g., the maximum 60 minutes is time 

measurement), and 

comparing the candidate object interval to a weighted measure of object density for the current object 
cluster, the measure of object density corresponding to a measure of distribution of distances separating adjacent 
ones of the objects in the current object cluster measured in the selected dimension of the context-related 
metadata (During clustering, the rate at which photographs were taken within a cluster is 
considered. For instance, while taking a hike through a forest, someone may take a picture 
every couple of minutes. In contrast, when photographing a newborn baby for the first time, the 
time between pictures is likely to be in seconds. Different rates are taken into account by 
comparing each pair of consecutive photographs to the basic photographic rate of the cluster. 
When a pair with a time difference that appears to be outside the normal range for the cluster, a 
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new cluster is created (Graham, Page 4, Col. 2 Lines 33-54). The Graham's teaching as 

diSCUSSed indicates the Step Of comparing the candidate object interval to a weighted measure of object 

density for the current object cluster, e.g., time interval of two consecutive images is compared with 

basic photographic rate Of the Cluster, the measure of object density corresponding to a measure of 
distribution of distances separating adjacent ones of the object in the current object cluster measured in the selected 

dimension of the context-related metadata, e.g., the basic photographic rate of the cluster such as 5 
sec between two consecutive photographs is a measure of distribution of distances separating 
the images adjacent to the current image in the cluster). 

Regarding claim 2, Graham teaches all of the claimed subject matter as discussed 
above With respect to Claim 1 , Graham further discloses the measure of cluster extent for each current 
object cluster corresponds to a temporal distance spanned by recorded generation times associated with all 
objects In the current object cluster (Graham, Page 4, Col. 2 Lines 3-28). 

Regarding claim 3, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 1, Graham further discloses the measure of cluster extent for each current 
object cluster corresponds to a spatial distance spanned by recorded generation locations associated with all 
objects In the current object cluster (Graham, Page 3, Col. 2 Line 36-Page 4, Col. 1 Line 2). 

Regarding claim 4, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 1 , Graham further discloses the measure of object density for each current 
object cluster corresponds to an average temporal distance separating adjacent objects in the current object 
cluster (Graham, Page 4, Col. 2 Lines 33-54). 
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Regarding claim 5, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 1 , Graham further discloses the measure of object density for each current 
object cluster corresponds to an average spatial distance separating adjacent objects in the current object cluster 
(Graham, Page 4, Col. 2 Lines 33-54). 

Regarding claim 6, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 1 , Graham further discloses the classifying comprises merging consecutive 
ones of the candidate objects into a current one of the object clusters until the candidate object interval 
determined for a current one of the candidate objects exceeds the weighted measure of cluster extent for the 
current cluster, at which point a successive one of the object clusters in the series is initiated with the current 
candidate object (Graham, Page 4, Col. 2 Lines 3-28). 

Regarding claim 7, Graham teaches all of the claimed subject matter as discussed 

above With respect to Claim 1 , Graham further discloses the classifying comprises merging consecutive 
ones of the candidate objects into a current one of the object clusters until the candidate object interval 
determined for a current one of the candidate objects exceeds the weighted measure of object density for the 
current object cluster, at which point a successive one of the object clusters in the series is initiated with the 
current candidate object (Graham, Page 4, Col. 2 Lines 33-54). 

Regarding claims 8 and 9, Graham teaches all of the claimed subject matter as 
discussed above with respect to claim 1, Graham further discloses the processing comprises 
determining the weighted measures of cluster extent by applying to the measures of cluster extent respective 
weights that decrease with increasing sizes of the respective object clusters (Graham, Page 3, Col. 2 Lines 
1-10). 
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Regarding claim 10, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 1 , Graham further discloses the step of customizing at least one of the 
weights applied to the measures of cluster extent based on an analysis of objects in the corresponding object 
cluster (Graham, Page 4, Col. 2 Lines 19-28). 

Regarding claim 1 1 , Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 10, Graham further discloses the step of scaling at least one of the 
weights applied to the measures of cluster extent based on a fractal dimension estimate of recorded time 
generation meta data associated with the objects in the collection (Graham, Page 5, Col. 1 Lines 8-15). 

Regarding claim 12, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 1 , Graham further discloses the step of customizing at least one of the 

weights applied to the measures of cluster object density based on an analysis of objects in the corresponding 

object cluster (Graham, Page 5, Col. 2 Lines 8-15). 

Regarding claim 13, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 12, Graham further discloses the step of scaling at least one of the 
weights applied to the measures of cluster extent based on a fractal dimension estimate of recorded time 
generation meta data associated with the objects in the collection (Graham, Page 5, Col. 1 Lines 8-1 5). 

Regarding claim 14, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 1 , Graham further discloses the step of comparing the object density of a 
candidate object cluster consisting of the current object cluster and the candidate object with the weighted 
measure of object density for the current object cluster (Graham, Page 4, Col. 2 Lines 33-54). 
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Regarding claim 15, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 14, Graham further discloses the measure of object density for each 
current object cluster corresponds to an average temporal distance separating adjacent objects in the current 
object cluster (Graham, Page 4, Col. 2 Lines 33-54). 

Regarding claim 16, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 14, Graham further discloses the measure of object density for each 
current object cluster corresponds to an average spatial distance separating adjacent objects in the current object 
cluster (Graham, Page 4, Col. 2 Lines 33-54). 

Regarding claim 17, Graham teaches all of the claimed subject matter as discussed 
above With respect to Claim 14, Graham further discloses the measure of object density for each object 
cluster corresponds to a moving average distance separating adjacent objects in the current object cluster 

(Graham, Page 4, Col. 2 Lines 33-54). 

Regarding claim 18, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 14, Graham further discloses the step of determining the weighted 
measures of cluster extent by applying to the measures of cluster extent respective weights that decrease with 
increasing sizes of the respective object clusters (Graham, Page 3, Col. 2 Lines 1-10). 

Regarding claim 19, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 1, Graham further discloses the step of processing each of the candidate 
objects sequentially beginning at a first end of the object sequence (Graham, Page 4, Col. 2 Lines 3-1 8). 
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Regarding claim 21, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 1, Graham further discloses the sequence to be segmented includes objects 
of the following types: text, audio, graphics, still images, video and business events (Graham, Abstract). 

Regarding claims 23 and 32, Graham teaches a method of organizing a collection of 
objects, comprising: 

segmenting objects from the collection into clusters (Graham, Page 4, Col. 2 Lines 3-54); 

extracting context-related meta data corresponding to object generation location associated with the 
objects and parsable into multiple levels of a name hierarchy (Graham, Page 3, Col. 2-Line 39->Page 4, 
Col. 1-Line 2, Page 4, Col. 1-Lines 8-13, Page 5, Col. 1-Lines 16-29); and 

assigning names to clusters based on the extracted context-related meta data corresponding to a level of 
the name hierarchy selected to distinguish segmented clusters from one another (Graham, Page 2, FIG. 1a 

and 1b). 

Regarding claim 27, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 23, Graham further discloses the context-related meta data corresponds to 
recorded information relating to country, city, and state of object generation (Graham, Page 3, Col. 2-Line 

39->Page 4, Col. 1-Line 2, the GPS location data of a taken photo, e.g., latitude and longitude, 
relates to country, city and state). 

Regarding claim 28, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 23, Graham further discloses the context-related meta data corresponds to 
both object generation times and object generation locations (Graham, Page 3, Col. 2 Line 36-Page 4, 
Col. 1 Line 2). 
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Regarding claim 29, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 23, Graham further discloses the step eft automatically naming objects in 
agiven cluster based on the name assigned to thegiven cluster (Graham, Page 5, Col. 1 Lines 16-28 and 
FIG. 1a-b). 

Regarding claim 30, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 29, Graham further discloses the objects in the given cluster are named 
automatically in accordance with a chronological ordering of the objects in thegiven cluster (Graham, Page 
5, Col. 1 Lines 16-28 and FIG. 1a-b). 

Regarding claim 31, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 29, Graham further discloses the step of storing objects in thegiven 

cluster in a tree structure organized by cluster and labeled in accordance with the assigned names (Graham, 

FIG. 4 and 1a-b). 

Regarding claim 33, Graham teaches a method of organizing a collection of objects, 
comprising: 

accessing a sequence of objects segmented into clusters each including multiple constituent objects 
arranged in a respective sequence in accordance with context-related meta data associated with the objects 
(Graham, Page 5, Col. 2 Lines 35-47); 

selecting for each object cluster at least two constituent objects representative of beginning and ending 
instances in the corresponding object sequence (Page 5, Col. 2 Lines 23-47 and Page 6, Col. 1 -Lines 
37-49); and 

in a user interface, graphically presenting the selected representative objects of each cluster without 
graphically presenting representations of unselected ones of the constituent objects of the clusters (The 
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process of creating summaries as discussed on Page 5, Col. 2 Lines 42-47 is displayed 
similarly as in FIG. 1a-b). 

Regarding claim 34, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, Graham further discloses the step of graphically presenting a 
selected one of the clusters as a stack of partially overlapping images representative of multiple objects in the 
selected cluster (Graham, FIG. 1a-b). 

Regarding claim 35, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 34, Graham further disclose the step of revealing an increased portion of 
a given one of the representative images in the stack in response to detection of a user-controlled display icon 
positioned over the given representative image (Graham, FIG. 1a-b). 

Regarding claim 36, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, Graham further discloses the step of presenting the selected 
representative objects with the spacing between adjacent ones of the selected representative objects in the same 
cluster smaller than the spacing between adjacent ones of the selected representative objects in different clusters 
(Graham, Page 5, Col. 5 Lines 35-47). 

Regarding claim 37, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, Graham further discloses the step of merging objects of one cluster 
into an adjacent cluster in response to user input (Graham, Pages 3-4, Calendar Browser). 

Regarding claim 39, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 37, Graham further discloses the objects of the one cluster are merged into 
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the adjacent cluster in response to user selection of an icon for merging the clusters (Graham, Pages 3-4, 
Calendar Browser). 

Regarding claim 40, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, Graham further discloses the step of presenting a graphical 
representation of distributions of objects in the clusters (Graham, FIG. 1 a-b). 

Regarding claim 41, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 40, Graham further discloses object distribution for a given cluster is 
presented as object instances plotted along an axis corresponding to a scaled representation of the context-related 
extent spanned by the given cluster (Graham, FIG. 5). 

Regarding claim 42, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 40, Graham further discloses the step of splitting a given cluster in 
response to user selection of a point in the representation of the object distribution presented for the given cluster 
(Graham, FIG. 5). 

Regarding claim 43, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 40, Graham further discloses the step of automatically splitting a given 

cluster into two or more clusters in response to user input (Graham, FIG. 5). 

Regarding claim 44, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 43, Graham further discloses the given cluster is automatically split into a 
user-selected number of sub-clusters (Graham, FIG. 5 and Page 5, Col. 2 Lines 14-47). 
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Regarding claim 45, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 43, Graham further discloses the given cluster is automatically split based 
on relative sizes of intervals between successive objects in the given cluster (Graham, Page 4, Col. 2 Lines 
3-84). 

Regarding claim 46, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 33, Graham further discloses the context-related meta data corresponds to 
object generation times (Graham, Page 4, Col. 2 Lines 3-18). 

Regarding claim 47, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 33, Graham further discloses the context-related meta data corresponds to 
object generation locations (Graham, Page 3, Col. 2 Line 36-Col. 3 Line 2). 

Regarding claim 48, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 33, Graham further discloses the segmented sequence includes objects of 
the following types: text, audio, graphics, still images, video, and business events (Graham, Page 4, Col. 2 
Lines 3-18). 

Regarding claim 49, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, Graham further discloses the step of graphically presenting at least 

one link to an object of a cluster arranged in a sequence in accordance with time-related meta data in a calendar 

format (Graham, FIG. 1a-b). 
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Regarding claim 51 , Graham teaches a system of organizing a collection of objects, 
comprising a user interface layout engine operable to perform operations comprising: 

a computer-readable medium storing computer-readable instructions; and a data processing; unit 

coupled to the memory, operable to execute the instructions (These features are inherited features of a 
computer system as disclosed by Graham on Page 5, Col. 2-Lines 2-13), and based at least in 
part on the execution of the instructions operable to perform operations comprising 

accessing a sequence of objects from the collection segmented into clusters each including multiple 
objects arranged in a respective sequence in accordance with context-related meta data associated with the 
objects (Graham, Page 5, Col. 2 Lines 35-47); 

selecting for each object cluster at least two constituent objects representative of beginning and ending 
instances in the corresponding object sequence (Graham, Page 5, Col. 2 Lines 23-47 and Page 6, Col. 
1-Lines 37-49); and 

in a user interface, graphically presenting the selected representative objects of each cluster on a screen 
without graphically presenting representations of unselected ones of the constituent objects of the clusters (The 

process of creating summaries as discussed on Page 5, Col. 2 Lines 42-47 is displayed 
similarly as in FIG. 1a-b), 

wherein the user interface layout engine presents the selected representative objects with the spacing 
between adjacent ones of the selected representative objects in the same cluster smaller than the spacing between 
adjacent ones of the selected representative objects in different clusters (Graham, Page 5, Col. 5 Lines 35- 
47). 
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Claim Rejections - 35 USC §103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 

claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 

claims was commonly owned at the time any inventions covered therein were made absent any 

evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 

the inventor and invention dates of each claim that was not commonly owned at the time a later 

invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 

and potential 35 U.S.C. 1 02(e), (f) or (g) prior art under 35 U.S.C. 1 03(a). 

Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Graham et 
al. [Time as Essence for Photo Browsing Through Personal Digital Libraries]. 

Regarding claim 20, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 19, Graham does not teach the step of processing each of the candidate 
objects sequentially beginning at a second end of the object sequence opposite the first end. However, 
clustering the image by iterating the sorted list at the other end of the list is not different from the 
first end. Therefore, it would have been obvious for one of ordinary skill in the art at the time the 
invention was made to iterate the sorted list at the other end in order to cluster the images. 
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Claims 38 and 50 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Graham et al. [Time as Essence for Photo Browsing Through Personal Digital Libraries]. 

Regarding claim 38, Graham teaches all of the claimed subject matter as discussed 
above with respect to Claim 37, but does not teach objects of one cluster are merged into an adjacent 
cluster in response to dragging and dropping of the objects to be merged. 

However, the Calendar Browser as disclosed by Graham must be implemented in a 
computer system with a conventional operating system such as Window XP (Graham, Page 1, 
Col. 2 Lines 12-21 ). By using Window XP, an object can be dragged and dropped from a folder 
to another folder. 

Therefore, it would have been obvious for one of ordinary skill in the art at the time the 
invention was made to include the feature of dragging and dropping of Window XP into the 
Calendar Browser in order to manipulate objects between folders. 

Regarding claim 50, Graham teaches all of the claimed subject matter as discussed 
above with respect to claim 33, but does not disclose the step of graphically presenting at least one 
link to an object of a cluster arranged in a sequence in accordance with location- related meta data in a map 
format. 

However, as suggested by Graham, the location where a photograph was taken could 
also be extracted form the image files and introduces as a clustering criterion (Graham, Page 3, 
Col. 2 Line 36-Page 4, Col. 1 Line 2). 

Thus by using location for clustering, the location is used to name a folder and the 
process of linking is similar to FIG. 1a-b. A folder naming by location indicates a map format. 
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Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing date 
of this final action. 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to HUNG Q. PHAM whose telephone number is 571-272-4040. The 
examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, JAMES K. TRUJILLO can be reached on 571-272-3677. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
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would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

HUNG Q. PHAM 
Primary Examiner 

/HUNG Q. PHAM/ Art Unit 21 59 

Primary Examiner, Art Unit 2159 
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